A Novel Evaluation Methodology to Obtain Fair Measurements in Multithreaded Architectures

Nowadays, multithreaded architectures are becoming more and more popular. In order to evaluate their behavior, several methodologies and metrics have been proposed. A methodology defines when the measurements of a given workload execution are taken. A metric combines those measurements to obtain a final evaluation result. However, since current evaluation methodologies do not provide representative measurements for these metrics, the analysis and evaluation of novel ideas could be either unfair or misleading. Given the potential impact of multithreaded architectures on current and future processor designs, it is crucial to develop an accurate evaluation methodology for them. This paper presents FAME, a novel evaluation methodology aimed to fairly measure the performance of multithreaded processors. FAME reexecutes all threads in a multithreaded workload until all of them are fairly represented in the final measurements taken from the workload. We compare FAME with previously used evaluation methodologies in architectural research scenarios. Our results show that FAME provides the most accurate measurements, becoming an ideal evaluation methodology to analyze novel design ideas implemented in multithreaded architectures.